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Non-developmental  Items  (NDI)  acquisition  programs  are  enjoying  popular 
support  as  faster,  cheaper  alternatives  to  new  start  research  and  development 
programs.  Unfortunately,  DoD  policy  on  risk  management  in  NDI  programs  is 
lacking.  In  the  new,  less-restrictive  DoD  Directive  5000.1  Acquisition  Policy 
and  DoD  Regulation  5000. 2-R,  the  program  manager  (PM)  is  expected  to  tailor 
risk  management  practices  to  the  needs  of  the  program.  Tailoring  DoD  risk 
management  policy  to  support  NDI  program  management  leaves  the  PM  too 
much  guesswork.  An  NDI  PM’s  risk  management  program  cannot  reasonably 
benefit  from  DoD  risk  management  guidance,  procedures,  and  tools  because 
he  or  she  is  focused  on  new  development  program  risks  and  risk  management 
practices.  Missing  is  any  explicit  consideration  of  unique  NDI  risks  and  risk 
management  requirements.  NDI  PMs  need  more  structured  policy  and 
instruction  regarding  NDI  risk  management  for  the  streamlined,  accelerated 
NDI  acquisition  environment.  A  lesson  that  we  have  learned  is  that  we  need  a 
published  risk  management  plan  as  the  source  of  NDI  risk  management 
program  decisions  and  actions.  This  article  makes  specific  recommendations 
for  including  NDI  risk  management  in  DoD  policy. 


Today’s  era  of  defense  downsizing 
has  the  military  services  scrambling 
to  protect  research,  development, 
and  acquisition  (RD&A)  funds  and  to  pre¬ 
serve  their  acquisition  programs.  Quality 
of  life  issues,  training  and  readiness,  and 
peacekeeping  missions  continue  to  draw 
funds  away  from  RD&A  programs  and 
modernization.  The  armed  services  and 
their  acquisition  program  managers  are 
challenged  to  justify  lengthy  and  costly 
new  research  and  development  programs 


when  commercial  or  nondevelopmental 
solutions  exist.  Risky  acquisition  strate¬ 
gies  and  mistakes  in  new  research  and  de¬ 
velopment  acquisitions  have  become 
harder  to  defend.  A  new  program’s  risks 
will  demand  close  management  to  prevent 
unanticipated  or  poorly  planned-for  events 
from  exposing  the  program  to  funding  cuts 
or  cancellation. 

To  the  rescue,  or  so  many  expect,  comes 
the  Non-developmental  Item  (NDI).  NDI 
system  acquisitions  give  PMs  and  the  gov- 
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Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 
VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 
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ernment  rapid,  lower  cost  access  to  ad¬ 
vanced  technology.  NDI  acquisition  strat¬ 
egies  offer  the  promise  of  risk  mitigation 
and  a  lower  risk  means  of  meeting  the 
armed  services’  urgent  mission  needs  and 
operational  requirements.  NDI  program 
management  is  not,  however,  without  risk. 
An  NDI  program  manager  must  tailor  the 
program’s  risk  management  to  the  unique 
risks  and  uncertainties  in  NDI  system  ac¬ 
quisitions.1  NDI  risk  management  de¬ 
mands  more  than  tailoring.  Problems  arise 
with  NDI  risk  management  because  NDI 
risks  and  necessary  risk  management  pro¬ 
cesses  are  not  well  understood  or  de¬ 
scribed  in  Depar  tment  of  Defense  (DoD) 
policy.  NDI  risk  management  demands 
policy,  framework,  and  tools  to  assist  the 
PM.  An  NDI  acquisition’s  puipose  is  to 
simplify  and  accelerate  the  acquisition 
process  to  meet  the  program’s,  and  ulti¬ 
mately,  our  soldier’s  needs.  Successful 
NDI  acquisition  programs  would  benefit 
from  a  more  clearly  mapped  risk  manage¬ 
ment  policy  structure,  rather  than  broad 
expectations  of  tailoring,  to  fulfill  their 
promise  of  simple,  rapid,  and  reduced-risk 
acquisitions.  The  simple,  fast,  cheaper 
route  to  NDI  risk  management  success 
requires  good  directions  and  signposts 
advising  PMs  of  risks  and  precautionary 
measures. 


Risk  Management  and 
Non-Developmental  Item  Overview 


DoD’s  risk  management  concepts  arc 
consistent  with  those  in  industry.  DoD 
guides  on  risk  management  are  similar  to 
the  Program  Management  Institute’s 
(PMI)  body  of  knowledge  on  risk  man¬ 
agement.  The  PMI  approach  reflects  the 
general  body  of  knowledge  on  risk  and 
risk  management  that  may  be  found  in 
business  and  industry  today. 

Risk  Definition 

Webster ’s  3d  International  Dictionary 
defines  risk  as  “the  possibility  of  loss,  in¬ 
jury,  disadvantage,  or  destruction;  some¬ 
one  or  something  that  creates  or  suggests 
a  hazard  or  adverse  chance;  the  product 
of  the  amount  that  may  be  lost  and  that 
probability  of  losing  it  (p.  1961).  DoD 
Directive  (DoDD)  5000. 1  and  DoD  Regu¬ 
lation  (DoDR)  5000.2-R  do  not  define  risk 
to  coincide  with  discussions  of  risk  man¬ 
agement  or  risk  reduction.  Risk  is  often 
characterized  by  type:  cost,  schedule,  tech¬ 
nical  performance,  supportability,  and  pro¬ 
grammatic  risk  (DSMC,  1989). 

These  five  types  of  risk  interact  to  af¬ 
fect  a  program’s  overall  performance. 


1  Tailoring  is  a  practice  that  means  modifying  the  acquisition  process  to  reduce  the  time  required  to  meet 
user  (soldier)  needs.  Tailoring  is  expected  to  be  done  according  to  common  sense,  effective  business 
practice,  laws,  regulations,  and  a  program’s  anticipated  fielding  date  (DoD  Directive  5000.1,  1996).  The 
concept  of  tailoring  risk  management  is  considered  valuable  by  DoD  as  part  of  the  program  manager’s 
authority. 


MAJ  Micheal  R.  Steves  is  Test  and  Evaluation  Coordinator  for  the  PATRIOT  Missile  System, 
Forward  Area  Air  Defense  Systems,  and  Task  Force  XXI  Tactical  Operations  Centers  at  Head¬ 
quarters,  U.S.  Army  Test  and  Evaluation  Command,  Aberdeen  Proving  Ground,  Maryland. 
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Their  basis  is  in  the  five-phase,  four-mile- 
stone,  full-scale  development  acquisition 
cycle  found  in  the  now-rescinded  DoD 
Instruction  5000.2. 

Risk  Management 

The  DoDD  5000.1  (1996,  p.4)  defines 
risk  management  as  an  approach  that  “en¬ 
compasses  risk  identification,  mitigation, 
and  continuous  tracking,  and  control  pro¬ 
cedures  that  feed  back  through  the  pro¬ 
gram  assessment  process  to  decision  au¬ 
thorities.”  This  definition  differs  from  the 
business  definition  in  that  it  stresses  a  sys¬ 
tematic  approach  rather  than  “art  and  sci¬ 
ence.”  Risk  management  identifies  and 
evaluates  the  program  areas  vulnerable  to 
high  levels  of  uncertainty.  Its  puipose  is 
to  provide  a  means  of  comparing  risk 
management  performance  to  a  standard 
and  of  tracking  risk-related  information. 
By  doing  so,  DoDR  5000.2-R  adds  that 
risk  management  becomes  an  essential  el¬ 
ement  of  a  program’s  acquisition  strategy 
(1996,  Part  3,  p.  3). 

The  Defense  Risk  Management  Process 

A  PM’s  risk  management  process 
should  set  objectives  and  continually  as¬ 
sess  the  program  for  obstacles  that  hinder 
accomplishment  of  those  objectives.  It  al¬ 
lows  the  PM  to  formulate  alternative  courses 
of  action  and  to  make  rational  decisions  on 
monitoring  and  controlling  the  outcomes  or 
consequences  of  program  events. 


Upon  identification  of  a  program’s 
risks,  a  four-part  process  begins.  The  four 
defense  risk  management  process  compo¬ 
nents  are  planning,  assessment,  analysis, 
and  handling.  Once  these  components  are 
implemented,  the  risk  mitigation  effort  is 
underway.  Risk  mitigation  is  the  combi¬ 
nation  of  risk  handling  and  risk  controls. 
DoDD  5000. 1  notes  that  technology  dem¬ 
onstrations,  prototyping,  and  test  and 
evaluation  are  the  available  techniques  to 
assess  and  handle  risk.  The  PM  should 
document  the  risk  management  program, 
process,  and  techniques.  Publication  of  a 
risk  management  plan  (RMP)  is  one 
means  of  accomplishing  this. 

Non-developmental  Items 

An  NDI  is  a  previously  developed  fed¬ 
eral,  state,  local,  or  foreign  government 
item  with  little  or  no  development  effort 
required  (5000.2-R,  Part  3,  p.5).  NDI  ac¬ 
quisition  strategies  look  to  governments 
or  commercial  vendors  as  sources  for  a 
developed  and  available  item.  DoDD 
5000. 1  encourages  the  armed  services  to 
use  the  most  cost-effective  materiel  solu¬ 
tion  over  a  system’s  life  cycle.  The  first 
choice  among  materiel  alternatives  after 
buying  commercial  off-the-shelf  items  is 
to  procure  or  modify  previously  developed 
U.S.  military  or  Allied  systems  or  equip¬ 
ment  (p.  6). 

The  armed  services  in  the  past  have  de¬ 
fined  NDIs  more  specifically.2  The  Army 


p 

Confusion  sometimes  results  from  the  attempt  to  distinguish  an  NDI  from  a  commercial  off-the-shelf  (COTS) 
item.  A  COTS  item  does  not  make  up  a  portion  of  NDIs.  COTS  items  are  commercial  hardware  or  software 
items  not  yet  modified  by  the  government,  items  that  are  in  the  commercial  inventory  or  production,  that 
have  proven  their  performance  in  a  similar  environment,  that  have  an  existing  support  structure,  have  an 
internal  support  structure,  have  an  internal  configuration  that  flows  with  commercial  changes,  and  generally 
are  integrated  with  other  hardware  and  software  items  to  become  part  of  a  system  or  subsystem  capability. 
An  NDI  therefore  might  be  described  as  a  noncommercial  off-the-shelf  item. 
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version  distinguishes  NDI  in  three  catego¬ 
ries  (Quindlen,  1989): 

1 .  Category  A  (basic  NDI).  No  modi¬ 
fications  are  required. 

2.  Category  B  (NDI  adaptation).  This 
category  is  sometimes  referred  to  as 
“Ruggedized  NDI.” 

3.  Category  C  (NDI  integration).  This 
category  is  sometimes  referred  to  as 
“Militarized  NDI.” 

Advantages  and  Disadvantages  of  NDIs 

NDIs  are  considered  a  means  of  miti¬ 
gating  program  cost  and  schedule  risks. 
Risk  mitigation  can  be  implemented  in  a 
shortened  or  streamlined  Program  Defi¬ 
nition  and  Risk  Reduction  Phase  and  En¬ 
gineering  and  Manufacturing  Develop¬ 
ment  (EMD)  Phase  of  the  acquisition  pro¬ 
cess.  The  advantages  of  NDIs,  according  to 
the  Government  Accounting  Office  (GAO), 
include  shorter  acquisition  time,  reduced 
cost,  proven  technical  performance  or  ma¬ 
ture  technologies,  and  simplified  contract¬ 
ing  procedures  involved  in  procuring  estab¬ 
lished  products.  Yet  NDI  acquisitions  do  not 
eliminate  risk  they  may  actually  introduce 
new,  significant  risks.  These  risks  will  be 
examined  shortly. 

An  effective  NDI  acquisition  program 
must  fulfill  the  service’s  needs  by  field¬ 
ing  mature  technology  that  contributes  to 
combat  readiness  while  satisfying  the 
user’s  expectations.  The  three  features  that 
attract  substantial  DoD  and  Congressional 
support  for  NDI  or  commercial  alterna¬ 
tives  but  challenge  the  PM  are: 

1 .  Meeting  user  requirements  based  on 
available  government  or  commer¬ 
cial  market  solutions. 


2.  Preparing  minimum  but  not  objec¬ 
tive  (“gold-plated”)  user  require¬ 
ments  to  support  lower  program 
costs  and  rapid  system  fielding. 

3.  Providing  DoD  with  greater  access 
to  state-of-the-art  technology  to 
keep  pace  with  changing  threats, 
emerging  technologies,  and  innova¬ 
tive  combat  systems. 

The  danger  to  NDI  programs  and  their 
effective  risk  management  is  in  forcing 
NDI  solutions  on  PMs  and  their  custom¬ 
ers  (the  user  community),  when  those 
NDIs  neither  meet  requirements  nor  tran¬ 
sition  effectively  make  the  transition  from 
current  government  or  commercial  appli¬ 
cations  to  battlefield  environments. 
Broadly  stated  DoD  risk  management 
policy  and  guidelines  then  contribute  to 
disadvantages  as  well  as  advantages  of 
buying  NDIs. 

The  disadvantages  of  NDIs  and  NDI 
acquisition  strategies  stem  from  the  new 
start  research  and  development  program 
paradigms  in  the  acquisition  community. 
A  developmental  item  PM  traditionally 
faces  cost,  schedule,  performance,  sup- 
portability,  and  programmatic  risks; 
NDI  program  risks  are  often  unique.  In 
reviewing  the  literature,  I  have  found  that 
the  following  areas  make  up  the  most 
pressing  potential  risks  facing  NDI  pro¬ 
gram  objectives:  a)  requirements,  b)  NDI 
acquisition  management  environment,  c) 
performance  specifications  and  nongov¬ 
ernment  standards  (S&S),  d)  test  and 
evaluation  (T&E),  and  e)  integrated  logis¬ 
tics  support  (ILS). 

The  NDI  PM’s  principal  disadvantage 
is  a  lack  of  both  DoD  policy  attention  paid 
to  the  above  NDI  program  risks  and  the 
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procedures  and  tools  to  manage  them.  This 
void  in  NDI-specific  risk  management 
guidance  leaves  the  PM  without  current 
references  for  structuring  a  risk  manage¬ 
ment  program  or  publishing  an  RMP. 
Meanwhile,  the  pressure  to  use  NDI  solu¬ 
tions  at  the  expense  of  a  service’s  original 
needs  and  requirements  may  yield  fielded 
items  that  aren’t  compatible  or  interoper¬ 
able  with  the  intended  operational  envi¬ 
ronment  or  battlefield  systems. 

The  Divergent  Paths 


The  crux  of  the  NDI  risk  management 
issue  is  that  the  paths  of  risk  management 
policy  and  NDI  policy  do  not  cross.  DoD 
policy  emphasizes  both  risk  management 
and  NDI  but  in  differing  directions.  Risk 
management  policy  in  the  DoD  5000  se¬ 
ries  keys  on  developmental  items  and  ac¬ 
quisition  strategics.  This  is  demonstrated 
in  the  traditional  cost,  schedule,  and  per¬ 
formance  risk  orientation  of  current  policy. 
NDI  policy  focuses  on  risk  mitigation 
gained  through  use  of  established  govern¬ 
ment  sources  of  supply.  Cost  and  sched¬ 
ule  benefits  aside,  no  description  or  in¬ 
struction  for  NDI  program  risk  manage¬ 
ment  appears  in  the  DoD  5000  series.  The 
5000.1  and  5000.2-R  policy  voids  exist 
within  the  expectation  that  the  program 
manager  will  “tailor”  (overlap,  combine, 
or  omit)  risk  management  requirements  to 
an  NDI  acquisition  strategy. 

NDI  acquisitions  streamline  acquisition 
policy  and  particularly,  risk  management 
policy,  and  result  in  tradeoffs  between  pro¬ 
gram  objectives.  While  NDI  programs 
may  mitigate  risks  in  cost  and  schedule, 
technical  performance,  supportability,  and 
programmatic  risks  may  be  heightened  by 


the  NDI-specific  risks  listed  earlier.  The 
consequences  of  NDI-specific  risk  events 
in  these  areas  can  have  tremendous  affect 
on  cost  and  schedule  in  the  long  run. 

NDI  PMs  currently  conduct  risk  iden¬ 
tification,  planning,  analysis,  assessment, 
handling,  and  documentation  according  to 
the  5000.1  and  5000.2-R.  In  doing  so,  the 
PM  applies  developmental  acquisition  risk 
management  policy  and  procedures  focus¬ 
ing  on  traditional  developmental  program 
risks  to  nondevelopmental  systems.  The 
point  is  that  many  NDI  acquisition  PMs 
must  “think  on  the  move”  without  explicit 
directions  or  road  maps.  Without  DoD  or 
service-specific  NDI  risk  management 
policy,  instructions,  templates,  or  unoffi¬ 
cial  emphasis  in  Defense  Systems  Man¬ 
agement  College  (DSMC)  risk  manage¬ 
ment  publications  or  instruction,  PMs  will 
continue  to  rely  on  gut  feelings,  develop¬ 
mental  lessons  learned,  and  incomplete 
references.  NDI 

risk  manage  "The  crux  of  the  NDI 
ment  planning  risk  m(inagemenf 

can  be  second-  jssue  js  that  the 
guessed  since  paths  of  risk  man- 
no  official  refer-  agement  policy  and 
ences  can  be  NDI  policy  do  not 

cited  to  justify  cross." 
risk  manage¬ 
ment  decisions.  Lack  of  published  DoD 
RMP  formats  increases  this  risk. 

There  are  reasons  for  this  policy  void. 
The  risks  in  NDI  acquisition  are  not  as 
widely  described  or  documented  as  the 
benefits.  The  absence  of  documentation 
and  description  is  due  to  lack  of  familiar¬ 
ity  by  PMs,  users,  and  DoD  overall  with 
the  NDI  acquisition  risks  in  competitive, 
commercial  (defense-  and  nondefense-re¬ 
lated)  industries.  These  industry  sources 
provide  DoD  with  a  growing  number  of 
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NDI  systems  not  initially  developed  to 
meet  a  specified  aimed  service  requirement. 
The  defense  acquisition  community’s  un¬ 
familiarity  with  NDI  risks  and  risk  man¬ 
agement  means  a  lack  of  understanding 
with  commercial  risk  management  prac¬ 
tices  and  standards.  Without  formal  risk 
management  policy  and  guideline  initia¬ 
tives  by  DoD  acquisition  participants,  de¬ 
finitive  policy  and  instruction  has  insuffi¬ 
cient  support  or  momentum  to  be  officially 
implemented. 

Merging  the  Two  Paths 


The  challenge  of  DoD  is  to  bridge  the 
gap  between  developmental  acquisition 
risk  management  policy  and  practices  and 
those  that  apply  specifically  to  NDI  risks. 
Effective  application  of  NDI-based  risk 
management  policy  and  practice  can  save 
NDI  and  possibly  COTS  PMs  valuable 
time,  money,  and  manpower  resources 
while  assisting  them  in  effectively  meet¬ 
ing  program  goals  and  objectives. 

Proposed  DoD  Policy  Revisions 

DoDD  5000.1  Risk  Management 
Policy.  DoDD  5000.1  contains  five  areas 
which  could  be  revised  to  make  its  risk 
management  guidance  suitable  for  NDI 
PMs.  The  first  area  is  the  risk  management 
subsection  of  Part  1  (p.  4).  It  presently 
states: 

[Program  mangers]  and  other  acqui¬ 
sition  managers  shall  continually 
assess  program  risks.  Risks  must  be 
well  understood,  and  risk  manage¬ 
ment  approaches  developed,  before 
decision  authorities  can  authorize  a 
program  to  proceed  into  the  next 


phase  of  the  acquisition  process.  Risk 
management  encompasses  identifi¬ 
cation,  mitigation  and  continuous 
tracking,  and  control  procedures  that 
feed  back  through  the  program  as¬ 
sessment  process  to  decision  authori¬ 
ties. 

This  guidance  lacks  reference  to  a  de¬ 
finitive  risk  management  planning  struc¬ 
ture.  Currently,  NDI  PMs  are  expected  to 
tailor  the  DoDD  5000. 1  risk  management 
guidance  to  their  NDI  systems  acquisi¬ 
tions.  The  following  could  be  added  for 
more  structural  content  to  both  NDI  and 
developmental  risk  management  planning: 

Each  milestone  decision  point  will 
include  a  review  of  the  updated  risk 
management  plan  (RMP)  and  mea¬ 
sures  taken  to  identify,  assess,  ana¬ 
lyze,  handle,  and  document  program 
risks.  Continuity  will  be  maintained 
between  the  RMP  and  discussions  of 
program  risks  in  the  Integrated  Pro¬ 
gram  Summary,  Acquisition  Plan, 
Systems  Engineering  Master  Plan, 
Test  and  Evaluation  Master  Plan,  and 
Integrated  Logistics  Support  Plan. 

The  RMP  could  be  modeled  after 
DSMC  Risk  Management  or  DSMC  Sys- 
tems  Engineering  Management  Guides’ 
RMP  formats  and  tailored  to  the  require¬ 
ments  of  the  individual  program.  It  would 
require  the  “teaming”  of  the  user  and  the 
PM  to  integrate  a  risk  management  focus 
that  supports  operational  need  and  require¬ 
ments  development.  It  could  then  be  moni¬ 
tored  through  the  Integrated  Product  Team 
Process. 

The  second  area  is  the  Event-Oriented 
Management  subsection  of  Part  2  (p.  6), 
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which  describes  “a  rigorous,  event-ori¬ 
ented  management  process  that  empha¬ 
sizes  effective  acquisition  planning,  im¬ 
proved  and  continuous  communications 
with  users,  and  prudent  risk  management 
by  both  the  government  and  industry.” 
This  statement  overlooks  risk  areas,  es¬ 
pecially  those  in  NDI  programs:  require¬ 
ments,  NDI  acquisition  management, 
T&E,  S&S,  and  ILS. 

The  DoD  4245.7-M,  published  in  1985, 
described  the  areas  of  risk  that  jeopardize 
achievement  of  successful  cost,  schedule, 
and  performance  objectives.  It  did  not 
describe  NDI  risks  acting  as  cost,  sched¬ 
ule,  or  performance  drivers.  The  Hierar¬ 
chy  of  Materiel  Alternatives  subsection 
immediately  following  Event-Oriented 
Management  currently  doesn’t  refer  to 
them  either.  Event-oriented  managers  are 
empowered  with  the  authority  to  make 
trade  offs  between  performance  and 
schedule  for  the  sake  of  cost  objectives. 
The  authority  to  trade  off  performance  or 
schedule  to  control  program  cost  risks  can 
imply  the  acceptance  of  force  readiness 
and  doctrinal  capability  gaps.  If  no  opera¬ 
tionally  effective  replacement  systems 
exist,  then  such  trade  offs  can  have  sig¬ 
nificant  impact  on  force  training  and  com¬ 
bat  effectiveness. 

The  following  could  be  added  to  reflect 
NDI  risks: 

NDI  programs  shall  examine  and 
track  the  effects  of  requirements, 
NDI  acquisition  management,  Test 
and  Evaluation,  Specifications  and 
Standards,  and  Integrated  Logistics 
Support  risks  on  cost,  schedule,  and 
performance  objectives.  Measures  of 
performance  for  risk  handling  shall 
be  proposed  in  the  RMP  concerning 


these  risks.  Trade-offs  shall  be  con¬ 
sidered  primarily  when  current  sys¬ 
tems  exist  to  sustain  the  force  until 
the  NDI  system  is  fielded. 

The  third  area  of  interest  is  a  combined 
look  at  the  Test  and  Evaluation  and  Mod¬ 
eling  and  Simu¬ 
lation  subsec¬ 
tions  in  Part  2 
(p.  6).  The  Risk 
Assessment  and 
Management 
subsection  re¬ 
fers  to  test  and 
evaluation.  In¬ 
tegrated  testing 
and  evaluation,  including  modeling  and 
simulation,  is  used  to  identify  areas  of 
technical  risk,  to  “assess  attainment  of 
technical  performance  parameters...  to  re¬ 
duce  the  time,  resources,  and  risks  of  the 
acquisition  process”  (p.  6).  The  Risk  As¬ 
sessment  and  Management  subsection  it¬ 
self  reads:  “To  assess  and  manage  risk, 
[program  managers]  and  other  acquisition 
managers  shall  use  a  variety  of  techniques, 
including  technology  demonstrations, 
prototyping,  and  test  and  evaluation. 

NDI  acquisition  strategies  often  include 
accelerated  and  streamlined  (depending  on 
the  quantity  and  quality  of  contractor  test 
programs  and  data)  testing  and  evaluation 
processes  or  simply  contractor  test  and 
evaluation  data  reviews.  The  risks  in  NDI 
acquisitions  are  testing  too  little,  too  much, 
and  testing  to  the  wrong  or  modified 
requirements  using  contractor  test  and 
evaluation  inputs.  The  following  could  be 
included  to  address  these  concerns: 

NDI  market  analysis  of  proposed  de¬ 
signs,  NDI  test  and  evaluation  pro- 


"The  risks  in  NDI 
acquisitions  are 
testing  too  little,  too 
much,  and  testing  to 
the  wrong  or  modi¬ 
fied  requirements 
using  contractor  test 
and  evaluation 
inputs." 
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grams,  and  NDI  performance  speci¬ 
fications  shall  carefully  consider 
trade  offs  in  test  and  evaluation  for 
the  sake  of  short-term  cost  and 
schedule  objectives.  Such  trade  offs 
introduce  risks  to  NDI  life-cycle 
costs  and  support.  NDI  contractors’ 
test  and  evaluation  data  shall  be 
screened  and  validated  indepen¬ 
dently  by  the  operational  test  and 
evaluation  community. 

The  fourth  area  considers  contractor  re¬ 
sponsibilities  again  from  the  Risk  Assess¬ 
ment  and  Management  subsection.  Previ¬ 
ously,  solicitation  documents  required 
contractors  to  identify  risks  and  specify 
plans  to  assess  and  eliminate  risks  or  re¬ 
duce  them  to  acceptable  levels.  Now, 
DoDD  5000. 1  directs  that  (p.  4)  “To  en¬ 
sure  an  equitable  and  sensible  allocation 
of  risk  between  government  and  industry, 
[program  mangers]  and  other  acquisition 
managers  shall  develop  a  contracting  ap¬ 
proach  appropriate  to  the  type  of  system 
being  acquired.” 

The  shortfall  in  this  guidance  on  con¬ 
tractor  risk  management  is  the  lack  of  par¬ 
allelism  between  the  NDI  PM’s  RMP  and 

requirements 
and  those  of  the 
contractor.  The 
NDI  risk  man¬ 
agement  efforts 
should  empha¬ 
size  teaming 
and  risk-sharing 
to  relieve  the 
burden  of  risk 
management 
from  resting  primarily  on  the  government. 
This  is  particularly  true  with  the  acceler¬ 
ated  nature  of  NDI  acquisitions.  Risk  re¬ 


sponsibility  in  NDI  programs  should  rest 
in  large  part  with  the  contractor,  given  the 
supposedly  mature  system  being  offered. 
The  following  could  be  added:  “Contrac¬ 
tor  RMPs  shall  be  specified  in  solicitations 
as  a  deliverable.  These  shall  be  consistent 
with  DoD  program  managers’  RMPs.  Risk 
sharing  shall  be  emphasized  in  contracts 
and  monitored  by  the  DoD-contractor 
team  through  integrated  product  teams 
(IPTs). 

The  fifth  area  combines  its  focus  on  the 
Tailoring,  IPT,  and  Management  Control 
subsections  in  Part  2  (p.  8).  The  guidance 
makes  no  mention  of  NDI  risks  even  as 
materiel  alternatives  arc  repeatedly  called 
for  prior  to  beginning  a  new  major  defense 
acquisition  program.  Risk  plans,  assess¬ 
ments,  and  controls,  which  are  subject  to 
tailoring,  should  be  comprehensive  in  na¬ 
ture  and  include  the  full  spectrum  of  risks 
expected  for  a  NDI  program.  Tailoring, 
IPTs,  and  management  control  could  ad¬ 
dress  the  five  NDI-specific  risk  areas  by 
directing  that  an  RMP  be  prepared  and  re¬ 
viewed  at  each  milestone. 

The  DoDD  5000.1  risk  management 
policy  revisions  are  an  important  first  step. 
Policy  must  clearly  state  the  risk  manage¬ 
ment  requirements  and  guidelines  as  they 
specifically  affect  a  NDI  PM’s  risk  man¬ 
agement  program.  These  changes  will  pro¬ 
vide  impetus  for  revisions  to  DoDR 

5000. 2- R,  which  will  be  examined  next. 

DoDR  5000.2-R  Risk  Management 
Policy  and  Procedures.  The  NDI  pro¬ 
gram  managers  need  more  definitive  and 
explicit  policy  and  procedures  in  DoDR 

5000.2- R  pertaining  to  their  risk  manage¬ 
ment  programs.  The  5000.2-R,  even  with 
its  increased  commercial  and  nondevelop- 
mental  item  emphasis,  neglects  NDI  risk 


"The  NDI  risk  man¬ 
agement  efforts 
should  emphasize 
teaming  and  risk¬ 
sharing  to  relieve 
the  burden  of  risk 
managment  from 
resting  primarily  on 
the  government." 
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management.  With  increasing  competition 
for  shrinking  DoD  RD&A  budgets  and  re¬ 
sources,  DoD  and  Congressional  risk  tol¬ 
erance  will  decline,  pressure  will  inten¬ 
sify  to  manage  programs  according  to 
sound  business  practices,  and  “safer” 
acquisitions  will  attract  more  support.  In 
these  circumstances,  explicit  NDI  risk 
management  instruction  will  become  criti¬ 
cal  to  NDI  program  managemen  t. 

Two  parts  of  DoDR  5000.2-R  could 
better  serve  NDI  PMs  with  more  explicit 
discussion  of  NDI  risks  and  risk  manage¬ 
ment  implementation  procedures.  The  first 
is  Part  3.  The  regulation  encourages  ac¬ 
quisition  program  stakeholders  to  exam¬ 
ine  the  full  range  of  mature  or  developing 
materiel  alternatives  before  initiating  a 
new  program.  Emphasis  is  placed  on  cost, 
schedule,  performance,  interoperability, 
trade  offs,  and  risk  management.  The  use 
of  IPTs  and  tailoring  are  introduced  as 
additional  major  program  considerations 
for  each  milestone  decision  point.  A 
program’s  decision  authority,  documenta¬ 
tion,  and  milestone  reviews  will  be  sub¬ 
ject  to  the  program  size,  complexity,  and 
risk,  as  well  as  the  flexibility  intended 
through  IPTs  and  tailoring. 

As  part  of  the  original  DoD  Instruction 
5000.2,  risk  assessment  formats  were  pro¬ 
vided  in  the  now-rescinded  DoD  5000.2- 
M.  The  format  description  did  more  to 
encourage  tailoring  than  to  depict  a  spe¬ 
cific  risk  assessment  format;  it  provided 
little  to  support  outlining  an  NDI  risk  as¬ 
sessment,  much  less  an  RMP.  The  current 
5000.2-R  provides  no  formats.  PMs  are 
given  wider  latitude  to  prepare  their  own 
risk  assessments  and  risk  management 
plans  and  programs.  Any  current  format 
would  be  helpful  if  it  were  to  refer  to  the 
the  old  DoD  4245. 7-M  risk  areas. 


Additionally,  with  NDI  and  streamlining 
initiatives  more  common  today,  more  de¬ 
tail  is  required  than  currently  described  or 
provided  in  those  sets  of  templates  in  the 
supportability  and  programmatic  sources 
of  risk  as  well  as  the  NDI-specific  Re¬ 
quirements,  S&S,  T&E,  ILS,  and  NDI 
program  management  environment  risk 
areas. 

In  the  5000.2-R,  milestone  0  concen¬ 
trates  on  materiel  alternatives  to  a  new 
major  defense  acquisiton  program.  The 
milestone  I,  II,  and  III  reviews,  however, 
repeatedly  focus  on  acquisition  program 
baselines  (APB),  acquisition  strategies, 
and  exit  criteria.  The  APB  content,  espe¬ 
cially  pertain¬ 
ing  to  cost,  is 
subject  to  re¬ 
finement  during 
each  program 
phase.  A  refined 
APB  encour¬ 
ages  risk  reduc¬ 
tion  efforts.  Risk  reduction  efforts  are 
based  on  careful  risk  assessments,  which 
are  a  basis  for  overall  cost  parameters  and 
realistic  cost  estimates.  But  no  format  is 
provided  in  the  regulation.  No  tie-in  is 
made  to  NDI  APBs  or  NDI  risk  assess¬ 
ments  and  unique  risk  reduction  require¬ 
ments.  Program  acquisition  strategics  each 
have  risk  management  as  an  essential  ele¬ 
ment.  Again,  while  NDI  programs  will 
have  their  own  acquisition  strategies  and 
risk  management  programs,  no  format  or 
tools  are  offered.  The  exit  criteria  or 
“gates”  that  the  system  must  pass  to  meet 
program  goals  and  enter  a  new  acquisi¬ 
tion  phase  are  selected  to  track  progress 
in  important  technical,  schedule,  or  man¬ 
agement  risk  areas.  No  explicit  mention 
is  made  of  NDI,  Requirements,  S&S, 


"The  targeted  risks 
are  technical,  manu¬ 
facturing,  and  sup¬ 
port  risks  that  must 
be  controlled  before 
the  next  milestone 
decision  point." 
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T&E,  ILS,  and  NDI  program  management 
risk  areas,  or  their  influence  on  exit  crite¬ 
ria. 

The  second  part  of  DoDR  5000.2-R 
procedures  concerns  Paid  1 .  The  milestone 
decision  point  reviews  discussed  in  DoDR 
5000.2-R  do  not  cite  one  central  document 
as  the  reference  for  a  program’s  risk  data 
or  the  planned  risk  management  program. 
Part  1  points  to  Phase  I,  Program  Defini¬ 
tion  and  Risk  Reduction,  as  the  period 
when  prototyping,  technical  demonstra¬ 
tions,  and  early  operational  assessments 
are  used  to  assess  and  reduce  risk.  The 
targeted  risks  are  technical,  manufactur¬ 
ing,  and  support  risks  that  must  be  con¬ 
trolled  before  the  next  milestone  decison 
point.  The  regulation  implies  that  these 
risk  reduction  techniques  are  done  prima¬ 
rily  in  Phase  I.  In  addition  to  emphasizing 
risk  management  as  a  program  function 
done  throughout  the  acquisition  life  cycle, 
reference  should  be  made  to  an  RMP  as 
the  source  of  documented  program  risks, 
risk  assessments  and  analysis,  risk  reduc¬ 
tion  measures,  rationale,  and  assumptions 
in  published  risk  ratings.  This  could  be 
more  effective  than  flipping  between  the 
Integrated  Program  Summary,  Acquisition 
Plan,  T&E  Master  Plan,  and  Systems  En¬ 
gineering  Master  Plan. 

The  regulation  policy  should  explicitly 
cite  the  NDI  streamlined  acquisition  cycle, 
and  the  parts  of  DoDR  5000.2-R  in  which 
detailed  descriptions  of  NDI  risks  and  risk 
management  actions  appeal-.  For  example, 
part  3.3.2  of  the  5000.2-R  could  feature 
the  NDI  requirements,  test  and  evaluation, 
integrated  logistics  support,  performance 
specifications  and  commercial  standards, 
and  NDI  acquisition  management  risks. 
Risks,  possible  risk  ratings  and  assessment 
considerations,  and  initial  risk  handling 


options  could  be  listed  as  program 
deliverables  at  milestone  reviews.  NDI 
risks  and  risk  management  could  then  be 
distinguished  from  the  developmental  fea¬ 
tures  in  the  DoDR  5000.2-R. 

Risk  Management  Templates.  Below 
are  areas  for  consideration  using  DoD’s 
former  DoD  4245. 7-M  risk  management 
tool,  the  risk  template.  This  would  clarify 
DoD’s  position  on  NDI  risks  and  provide 
recommended  risk  management  measures 
to  the  benefit  of  NDI  PMs,  their  programs, 
and  their  customers. 

Requirements  Risk  Template 

Areas  of  Risk: 

1.  Operational  and  design  require¬ 
ments  that  are  ill-defined  or  over¬ 
look  NDI  alternatives. 

2.  Inadequate  market  analysis  that 
contributes  to  “mix  and  match  tech¬ 
nologies”  being  required  rather  than 
verification  of  what  technology  ac¬ 
tually  exists. 

3.  Trade  offs  in  threshold  and  objec¬ 
tive  performance  requirements  that 
fail  to  meet  the  user’s  stated  need. 
Improperly  defining  the  proposed 
system’s  prospective  sources  (com¬ 
mercial,  modified  commercial,  or 
NDI)  and  required  future  design 
modifications. 

Outline  for  Reducing  Risk: 

1 .  Insert  NDI  market  analysis  into  the 
acquisition  cycle  as  part  of  Phase 
0,  Concept  Exploration  and  Devel¬ 
opment. 
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2.  Use  the  IPT  structure  to  better 
screen  and  develop  requirements. 

Timeline:  All  phases. 

Performance  Specifications  and  Non¬ 
government  Standards  Risk  Template 

Areas  of  Risk: 

1 .  Technical  performance  in  commer¬ 
cial  applications  as  specified  in 
commercial  item  descriptions 
(CIDs)  may  not  equate  to  or  explic¬ 
itly  meet  technical  performance  in 
military  applications  as  stated  in 
MIL-SPECS  and  MIL-STDS  after 
an  NDI  acquisition  strategy  is  al¬ 
ready  approved. 

2.  Performance  specifications  and 
standards  based  on  form.  fit.  and 
function  (that  allow  contractors  to 
design  solutions)  instead  of  the 
“how-to”  MIL-SPECS  and  MIL- 
STDS  used  in  design  and  manu¬ 
facturing  may  encounter  workforce 
resistance  or  complacency. 

3.  Inadequate  market  analysis  leads  to 
acceptance  of  products  having  in¬ 
sufficient  or  undocumented  techni¬ 
cal  data  or  CIDs  with  which  to  re¬ 
compete  the  procurement  for  future 
buys. 

Note:  The  DoDR  5000.2-R  states  that 
the  PM  “shall  structure  the  acquisition 
strategy  to  promote  sufficient  program  sta¬ 
bility  to  encourage  industry  to  invest,  plan, 
and  bear  risks  ...  program  acquisition  strat¬ 
egics  must  analyze  industrial  capability  to 
design,  develop,  produce,  support,  and 


restart  a  program...  analysis  will  identify 
DoD  investments  needed  to  create  any 
new  industrial  capabilities  and  the  risk  of 
industry  not  providing  the  manufacturing 
capabilities  at  the  planned  cost  and  sched¬ 
ule”  (Part  3,  p.  5). 

Outline  for  Reducing  Risks: 

1 .  Specify  in  solicitations  that  CIDs  for 
meeting  user  requirements  are  a 
deliverable. 

2.  Evaluate  the  CIDs  against  MIL- 
SPECS  to  verify  their  adequacy  for 
design  and  development. 

3.  Train  and  educate  the  acquisition 
workforce  in  CIDs  and  commercial 
specifications. 

Timeline:  All  phases. 

NDI  Acquisition  Managemen  t  Risk 

Template 

Areas  of  Risk: 

1 .  Despite  streamlining,  paperwork  re¬ 
quirements,  pricing  data,  account¬ 
ing  requirements,  and  continuous 
audits  of  NDI  programs  stifle  the 
cost  and  schedule  objectives  laid  out 
in  the  acquisition  strategy. 

2 .  Traditional  developmental  program 
paradigms  and  developmental  pro¬ 
gram  mindsets  continue  to  reflect  a 
cultural  resistance  toward  imple¬ 
menting  timely  and  cost-effective 
NDI  acquisition  strategies. 

3.  Lackof  PM-contractor  andPM-user 
teaming  on  risk  responsibility  and 
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risk  sharing  hampers  flexibility  in 
risk  management  efforts. 

4.  Programmatic  micro-management 
by  stakeholders  defeats  the  benefits 
of  an  NDI  acquisition  strategy. 

Note:  DoDR  5000.2-R  discusses  cost 
management  incentives,  stating  “risk  re¬ 
duction  through  the  use  of  mature  pro¬ 
cesses  shall  be  a  significant  factor  in 
source  selection  (Paid  3,  p.7)  ...  and  “the 
acquisition  strategy  shall  discuss  types  of 
contracts  ...  to  include  considerations  of 
risk  assessment,  reasonable  risk  sharing 
by  government  and  contractor(s)”  plus  the 
schedule  risk  of  using  government  fur¬ 
nished  equipment  or  government  fur¬ 
nished  information  (Part  3,  p.  8). 

Outline  for  Reducing  Risks: 

1.  Require  RMPs  as  a  contractor  de¬ 
liverable. 

2.  Require  workforce  training  and 
education  in  NDI  through  DSMC. 

Timeline:  All  phases. 

Test  and  Evaluation  Risk  Template 

Areas  of  Risk: 

1.  Requirements  are  not  stable,  realis¬ 
tic,  or  well-understood  by  designers, 
developers,  testers,  or  managers. 

2.  Overtesting  conducted  despite  the 
presence  of  satisfactory  contractor 
test  and  evaluation  data  package. 

3.  Developmental  and  technical  test¬ 
ing  costs  are  saved  but  operational 


testing  for  operational  effectiveness 
and  suitability  may  involve  condi¬ 
tions  not  grasped  by  contractor  test¬ 
ing  program.  These  incomplete  tests 
and  data  may  be  overlooked  or 
unquestioned  in  the  accelerated 
NDI  acquisition  cycle  and  corre¬ 
sponding  accelerated  NDI  testing 
program. 

Outline  for  Reducing  Risks: 

1 .  Test  and  evaluation  data  reviews  of 
contractor  commercial  testing  pro¬ 
gram  and  results. 

2.  Demonstrations  of  the  contractor’s 
testing  process. 

3.  Modeling  and  simulation  anchored 
in  realistic,  integrated  T&E  with 
combined  DT/OT,  and  live  fire 
T&E.  Examples  are  TECOM’s 
Simulation  and  Modeling  Anchored 
in  Real  Testing  (SMART)  program 
and  Virtual  Proving  Ground  initia¬ 
tives  and  its  Combat  Synthetic  Test 
and  Training  Assessment  Range 
(STTAR)  capability  used  at  a  recent 
National  Training  Center  rotation  by 
the  1st  CAV  Division. 

Timeline:  All  phases. 

Integrated  Logistics  Support  Risk 

Template 

Areas  of  Risk: 

1 .  Technical  data  packages  may  be  un¬ 
available  or  incomplete,  which  cre¬ 
ates  instability  of  spares  and  parts 
access. 
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2.  Competitive  re-procurements  of 
parts  may  not  contain  proper  incen¬ 
tives  to  attract  spares  and  parts  ven¬ 
dors. 

3.  ILS  and  a  system  life-cycle  focus 
may  be  overlooked  during  the 
requirements  development  stage. 

4.  Use  of  military  standard  and  non¬ 
standard  parts  creates  multiple  parts 
and  spares  lines. 

5.  Depot  and  repair  levels  may  not  be 
defined  in  terms  of  operational  en¬ 
vironments. 

Outline  for  Reducing  Risks: 

1 .  Define  ILS  requirements  when  de¬ 
ciding  what  category  of  NDI  the  ac¬ 
quisition  strategy  involves. 

2.  Conduct  market  analysis  of  contrac¬ 
tor  ILS  capabilities,  ILS  testing,  and 
support  demonstrations  in  the  in¬ 
tended  operational  environment  and 
conditions.3 

3.  Specify  training  packages  and  pub¬ 
lications  as  a  contractor  deliverable. 

Timeline:  All  phases. 

These  revisions  provide  guidance  to 
NDI  PMs.  The  template  diagram,  area  of 
risk,  risk  reduction  outline,  and  a  life  cycle 


timeline  for  managing  these  risks  should 
be  included.  Clarifying  risk  management 
policy  and  upgrading  NDI  risk  manage¬ 
ment  templates  can  streamline  the  risk 
management  process  by  saving  time,  man¬ 
power,  and  the  resources  required  to  staff 
a  risk  management  program.  The  NDI 
interpretation  of  developmental  item  tem¬ 
plates  would  become  unnecessary. 

Conclusions 


The  practice  of  risk  management  does 
not  benefit  from  “cookbook”  solutions.  If 
such  solutions  existed  for  developmental 
acquisition  programs,  there  are  very  few, 
if  any,  for  NDI  acquisition  programs.  This 
is  the  NDI  PM’s  dilemma.  The  NDI  PMs 
must  adhere  to  DoDD  5000. 1  and  DoDR 
5000. 2-R  risk  management  policies  in 
their  programs,  but  the  material  is  vague 
or  must  be  tailored  to  support  NDI  pro¬ 
gram  needs.  The  situation  leaves  the  NDI 
PM  driving  a  risk  management  program 
without  the  benefit  of  signposts  or  road 
maps. 

NDI  PMs  continuously  manage  risk  as 
part  of  today’s  streamlined  and  tailored 
NDI  acquisition  environment.  They 
should  expect  and  receive  succinct,  ex¬ 
plicit  policy  and  guidelines  to  help  them 
meet  their  risk  management  and  program 
management  goals  and  objectives.  DoDD 
5000.1  could  better  serve  the  NDI  PMs 
with  risk  management  language  directed 
to  their  specific  type  of  programs  and  ac- 


3 

Options  the  program  manager  can  consider  include  those  posed  in  DSMC’s  NDI  acquisition  publication 

1 .  Buy  commercial  upgrades  as  they  evolve  and  become  available. 

2.  Make  a  one-time  mass  spares  purchase  to  sustain  the  duration  of  the  system’s  life  cycle. 

3.  Buy  the  technical  data  package  to  solicit  sources  of  supply  that  coincide  with  the  end  of  original  production 
and  support  by  the  original  contractor. 
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quisition  strategies.  DoDR  5000.2-R  could 
provide  NDI  PMs  with  both  better  risk 
management  guidance  and  implementa¬ 
tion  procedures  with  NDI-based  instruc¬ 
tions.  formats  (once  found  in  DoD  5000.2- 
M),  and  tools  (once  found  in  DoD  4245.7- 
M  risk  templates  and  risk  management 
plan). 

The  NDI  program  manager’s  chal¬ 
lenges  and  program  risks  in  the  late  1990s 
are  not  adequately  reflected  in  DoD’s  mid- 


1980s  risk  management  policy,  proce¬ 
dures,  or  tools.  Risk  management  will  re¬ 
ceive  more,  not  less,  emphasis  as  an  ex¬ 
plicit  management  function.  NDI  and  NDI 
acquisition  strategies  will  continue  to 
grow  in  popular  support  as  DoD  RD&A 
budgets  are  “downsized”.  Modernization 
at  minimum  risk,  therefore,  will  require 
properly  marked  signposts  and  a  good 
road  map.  It  is  time  for  DoD  to  print  and 
distribute  those  signs  and  maps. 
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